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ABSTRACT 

The ways in which principals use performance 
appraisal data for personal professional development, the 
characteristics of those who use such information, and the features 
of performance appraisal systems and decision contexts that are 
predictors of appraisal use were studied through a scries of 
interviews at four sites (two with high appraisal users and two with 
low appraisal users). In all, 37 administrators, vice principals, and 
parents were interviewed. In general, the use of performance 
appraisal data was low to moderate. The positive nature of 
communication of appraisal results and the congruence of these 
results with principals' expectations had moderate positive 
influences on the use of data in decision making. Also important were 
decision context varieUDles corresponding to principals' experiences 
and knowledge, and the significance of the decision. Motivation for 
professional growth and attitude toward the appraisal were the most 
influential factors on learning about performance improvement. 
Appraisal systems for principals who art expert problem solvers might 
concentrate on improving aspects of the appraisal itself. Systems for 
moderately effective principals might be better concerned with 
decision setting variables that negatively influence use. (SLD) 
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... its difficult ... being in something for a number of years, to make drastic 
changes. Not that anything drastic is needed.... I guess from past experience 
very little is going to change because of it [the appraisal). (Principal. Site 
LUl) 

But if I have really thought about my strengths and weaknesoes? and the 
strengths and weaknesses within the school for that year, then we [principal 
and appraiser] can get into a more meaningful discussion that I think will lead 
to some definite plans for future [action].... I tended to direct it [appraisal 
process] through the kinds of paperwork I got ready and the information 1 
wrote down and gave to him [appraiser] to read, I believed in the process. It 
was basically my process. (Principal, Site HU2) 

These remarks ft-om two different principals were excerpted from transcripts of 
interviews regarding perceptions about appraisal practices concerning their own 
performance. Together they illustrate a considerable range in such views. They aiso beg 
the questions: (1) to what extent do principals use information concerning their own 
performance? and (2) how do (a) their perceptions, beliefs, attitudes and other "internal 
processes", (b) characteristics of the settings in which they work, and (c) features of the 
appraisal systems influence their use of such data for the purpose of performance 
improvement? 

In business and industry, where performance outcomes are relatively tangible 
(e.g., monetary profit margins, sales figures) and professionals usually aspire to vertical 
career progression, there is a need to rank or compare individuals according to 
productivity for purposes of promotion, remuneration compensation, and the like. 
However, in educational organizations (among some other publicly funded agencies) 
there is far less emphasis on vertical career progression. Consequently, appraisal 
systems are more likely to be geared to the assessment of current work rather than the 
potential for different kinds of work in the future. Decisions about matters such as pay 
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compensation are generally based on considerations other than performance: educational 
attainment and seniority are examples of such considerations (Ondrack & Oliver, in 
press). Nonetheless, the design, development, and implementation of performance 
appraisal systems in education has been stimulated in recent years largely by public 
demands for accountability. These systems are most often used by administrators to 
ensure that educators are not operating below minimum levels of competence and for the 
purposes of making administrative decisions (e.g., selection, tenure, promotion). But they 
also hold considerable promise for improving performance where improvement is defined 
by movement or professional growth from current performance levels toward 
organizational (government, system- school) goals or conceptions of effective practice. 
When performance appraisal is used for the purpose of professional development, the- 
primary "users" of the data become the individuals being evaluated. 

Recent research in education (e.g., Lawton, Hickox, Leithwood & Musella, 1986) 
suggests that although considerable time and effort are being devoted to personnel 
evaluation, its impact has been distressingly limited; this is so particularly when impact 
is defined by improved performance. These findings underscore the need to study the use 
for performance improvement of appraisal data by educators. In order to improve the 
benefits of performance appraisal systems such research should identify factors 
explaining the use of appraisal data. 

The present paper, which is based on data from a larger study (Cousins, 
forthcoming), addresses such questions as they apply to school principals. Principals 
were selected for study for two reasons. First, much of the data about knowledge 
utilization in general, and evaluation utilization in particular, is associated with senior 
managerial personnel in various organizations. However, managers, such as principals 
are typically responsible for directing most of the core, technical activities within their 
organizations. Very little is presently known about factors influencing the use of 
information in relation to such technical activities. Second, the actions, behaviors and 
practices of principals have been shown to explain a substantial proportion of the 
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observed variation in school effectiveness (e.g., Leithwood & Montgomery, 1982; 1986). 
As yet, however, relatively little is known about why they act as they do. Systematic 
study of their internal mental states and processes is a promising way to improve such 
understanding. 

To what extent and in what ways do principals use performance appraisal data 
for the purpose of personal professional development? What characteristics differentiate 
principals that are high users of such information from those who are not? What 
features of performance appraisal systems and decision contexts predict their use of 
appraisal data? These questions provide the central focus for the present paper. 

Framework 

The framework used for this study (see Figure I) has been described in detail 
elsewhere (Cousins & Leithwood, 1986; Cousins, forthcoming). This framework is 
grounded in a considerable body of empirical research about the utilization of evaluation 
data* Evaluation is generally defined in broad terms that incorporate a wide range of 
purposes as well as objects of evaluation. Because performance appraisal can be viewed 
as a special case of evaluation, it follows that knowledge generated about the use of 
evaluation data has considerable relevance to questions about the use of performance 
appraisal data« 

Conventional conceptions of knowledge utilization (e.g., Alkin, Daillak & White. 
1979; Weiss, 1981) relied on a continuum that ranged from instrumental uses (support 
for decisionmakers' decisions) to uses for conceptual development (the contribution of 
knowledge to user learning or educational outcomes). As both Alkin et al. and Weiss 
observed, traditional research on use clearly favored the instrumental conception and 
this severely restricted the range of observed uses of knowledge. Patton and his 
associates (Patton et aL, 1977) were among the first to recognize the inadequacy of 
conceptualizing use in strictly instrumental terms. This recognition has stimulated 
research on use considerably. 

ERLC ^ 
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Evaluation Utilization Conceptual 
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More recently, an even broader conception of the utilization process (e.g.. Cousins 
& Leithwood, 1986; Huberman. 1987; Kennedy, 1983; 1984), has been suggested - one 
that defines use as information processing by the user. According to this conception the 
mere psychological processing of evaluation data, without necessarily informing 
decisions, dictating actions or changing thought processes, constitutes use. 

The framework for this study incorporated all three types of use in its definition of 
the dependent variable. Evaluation data could be used to support discrete decisions 
(e.g., staffing, program management, program funding) or to educate decisionmakers 
about aspects of the ob;ect of evaluation (e.g., nature of program impact, components of 
programs explaining outcomes, etc.). Prior to either type of use, however, data must 
first be cognitively processed by decisionmakers (e.g., given. serious consideration) which 
might also result in it being discarded from further attention. 

Factors in the framework are defined as the circumstances or conditions that 
influence the extent to which evaluation data are used. The array of factors appearing 
in Figure 1 was derived from an extensive review of the empirical literature (65 studies) 
concerning the utilization of evaluation data. The studies examined the influences of 
factors that could be categorized according to one of two major hypothetical dimensions: 
characteristics of the evaluation implementation and characteristics of the decision or 
policy setting. These dimensions are correlated and interact with one another to produce 
effects (inhibiting, stimulating) on use. Six factors are encompassed by each of the two 
hypothetical dimensions. Those associated with the evaluation implementation are': 

• Evaluation quality . Characteristics of the evaluation process including the 
sophistication of methods, rigor, and type of evaluation model. An 
evaluation that attempts to link program components to program outcomes, 
for example, is considered to be more sophisticated than one that merely 
describes outcomes. 
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• Credibility , Credibility of the evaluator and'or the evaluation process defined 
by belie vability, objectivity, appropriateness of evaluative criteria, and the 
like. A well-seasoned evaluator with a proven track record is attributed 
higher levels of credibility than a novice, for example. 

• Relevance . The relevance of the evaluation to the information needs of 
decisionmakers according to: (I) the purposes of the evaluation and (2) the 
organizational location of the evaluator. Do the purposes of the evaluation 
meet the explicit and implicit needs of the audience(s) for whom the 
evaluation is conducted? Do evaluators working within the organization 
tend to produce evaluations that are more relevant? 

• Communication quality . Quality and/or clarity of the dissemination of results 
to the evaluation audience(s) according to characteristics such as the style 
of the report and the propensity of the evaluator to advocate its results. For 
example, is the report presented orally and/or in written form and does the 
evaluator follow-up the presentation with clarification? 

• Findings . The nature of the evaluation findings was defined by their 
positive or negative valence (e.g., judgments about whether the program is 
meeting its objectives), their consistency with the expectations of the 
evaluation audience(s), their value for decisionmaking, and the like. To 
what extent are findings predictable to decisionmakers? 

• Timeliness , The point in time at which* evaluation results are disseminated 
to decisionmakers relative to impending decision(s). Are the results 
presented too late to have an impact on the decision process? 

Factors associated with the decision or policy setting are: 

• Information needs . The type of information sought, number of evaluation 
audience(s) with differing information needs, time pressure and perceived 
need for evaluation. To what extent are explicit and implicit needs for 
evaluation information shared among different audiences? 

• Decision characteristics : Characteristics of decisions associated with the 
evaluation problem including decision impact area, type of decision, 
program novelty and significance of the decision, among other examples. 
Decisions regarding politically sensitive or controversial issues are of 
relatively high significance. 
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• Political climate . Characteristics associated with political climate such as 
political orientation of commissioners of the evaluation, dependence of 
decisionmakers on external sponsors, inter- and intra-organizational 
rivalries, budget fights and power struggles. Is it politically prudent for 
decisionmakers to decide in a manner that is consistent with the evaluation 
results? 

• Competins information . Information from sources beyond the evaluation 
relevant to the research problem and competing with evaluation data to 
inform decisions. Personal experience, informal observations made by 
decisionmakers and working knowledge are examples. 

• User personal characteristics . Decisionmakers' organizational role, 
information-processing style, organizational experience, and social 
characteristics, among other variables. Decisionmakers who carefully plan 
for the future and take preventative actions are distinguished from "crisis 
managers" who operate on more of a "reactive" basis. 

• User commitment and/or receptiveness to evaluation . Extent to which 
decisionmakers are open-minded about decisions and the evaluation 
findings. Are the decisionmakers dogmatic about the decision? Are they 
predisposed to attitudes about the utility of evaluation? 

The framework was used to help better understand principals' use of performance 
appraisal data defined quite broadly. All systematic attempts by appraisers to gather, 
analyze and interpret data relevant to principals' performance and to provide principals 
with the results of such assessments were included in the definition. This broad 
definition incorporated such supervisory practices as informal observations and 
visitations throughout the school year, requests for (collection oO information concerning 
the appraisee, responses to principals' requests for advice, communication of results or 
"feedback" to principals, and the like. 

According to the framework, the dependent construct of use was defined by 
principals basing discrete decisions (e.g., staffing, instructional leadership, student 
placement) on appraisal data or learning about their performance from the information 
communicated to them. However, prior to the occurrence of either type of use the data in 
the form of informal verbal feedback, written memos, post-appraisal conferences or other 
modes of communication were cot^nitivel v processed by principals, 
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Method 

A series of interviews were conducted in a single school system at each of four 
sites: two schools where principals had previously been designated "high appraisal data 
users'* and two schools where principals had been designated "low" on this dimension,^ 
The designations were made by a senior administrator in the system. At each site, the 
principal, the senior administrator responsible for the appraisal, the vice principal (if 
applicable), some teachers (selected within sites both randomly and in consultation with 
the principal) and parents with some knowledge of the principal's behavior were 
interviewed. Interview questions were refined after each of four rounds of data collection. 
After the first three rounds, repeat interviews with principals were conducted. Finally, 
documents relevant to the appraisals (e,g„ managers letters, summative letters of 
appraisal) were examined as an additional source of data. A total of 37 interviews were 
conducted and each was trascibed verbatim from audio tape. The transcripts and 
archival data were then content analyzed and the results were summarized both within 
and across cases. ^ 

The extent to which performance appraisal information was used by principals 
was typically low, or at best, moderate (see Table !)• To a modest extent, the appraisal 
results appeared to be taken seriously, or to have been cognitively processed by 
principals. Though the tendency to consider (process) appraisal information did not 
differ across sites, it appeared that some appraisal-based decisionmaking was more 
prevalent in sites identified pnor to data collection as high use sites. In most cases, this 
was evident in decisions to continue with current administrative directions. One 



For the purposes of the present discussion the codes "HU" and "LU" are used to identify hisfh and low 
use sites, respectively. 



"Further details regarding the methods used in this study for data collection, processing and analysis are 
provided by Cousins (forthcoming). For the purposes of the present discussion, suffice it to say that the 
methods and procedures developed by Miles and Fiuberman (1984) were followed reasonably closely, by that 
is meant, the framework described above significantly guided the collection, coding (content anal v sis) and 
display of data. 
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Table 1 

Types and Extent of Use Across Sites 



TYPE OF 
USE 



Low Use 
SiteLUl SiteLU2 



Site HUl 



High Use 

Site HU2 



Processing 



mod-high 



mod 



mod-low mod-high 



Education 



low 



low 



mod-low mod-low 



Decision 



mod-low mod-low 



mod 



mod-high 
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principal (Site HU2) commented that "I communicated all those intents to [the 
superintendent] and he agreed with me that Tes those were worthwhile activities and 
yes that would be a good thing to pursue'". Some principals, however, made decisions to 
change their directions based upon appraisal information. A decision regarding career 
planning (Site HU2) was an example. Based on feedback from her superintendent the 
principal decided to pursue certification requirements for supervisory officer positions. 

Very little appraisal-based learning concerning principal performance patterns 
was reported. One principal, for example, in a low-use site (LUl) paid attention to -< A 
fully understood what was communicated to him but subsequently based decisions on 
other information without learning much, if anything, about improving his performance. 

I think it [appraisal process] was a confirmation of what he already thought 
and felt all along. All we did was confirm each other's impressions in that 
experience.,,. There are not really manv surprises, (Superintendent, Site LUl). 

Analysis of observed patterns of use revealed that several factors had substantial 
influences. Many of these were associated with decision or policy setting dimension 
variables that characterized the principals themselves. In particular, principals' 
experience and working knowledge and motivation to grow professionally were 
antecedent variables that were found to be important predictors of use. They, in turn, 
influenced intervening variables such as principals' needs for information, attitudes 
toward the appraisal process, willingness to share information with appraisers and 
expectations regarding appraisal data. Other variables associated with the appraisal 
process, such as its rigor and aspects of the communication process, were also found to 
affect use. These variables differentially affected various types of use (processing, 
education, decision): such variations in influence are discussed below, 

Processins Appraisal Dati, 

As shown in Figure 2, principals' experiences and working knowledge were found 
to have moderate and negative effects on their propensity to give serious consideration to 
appraisal data. For the most part, this influence was indirect and became manifest 
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through other intervening factors. One such factor was principals* willingness to share 
information with appraisers. Principals with considerable experience in the role were 
less likely (although not exclusively) to be so inclined. They tended to minimize keeping 
their appraisers informed of issues and events in the school that had relevance to their 
own performance. The result was depressed data processing by the principals by (1) 
reducing the opportunities for ongoing communication (and ultimately the immediacy of 
evaluative feedback) between appraiser and appraisee and (2) impairing the 
meaningfulness of communication between appraisers and principals when evaluative 
information was given. EVincipals' willingness to share information with the appraiser 
turned out to be a fairly strong predictor of the extent to which principals paid attention 
to appraisal data. Following, is an example of criteria- used by a principal to decide \^hen 
to approach his appraiser (this principal was not particularly motivated to share 
information concerning his own performance): 

... if I think the parent is going to call him [superintendent] am I going to call 
him first? I suppose the answer is 'yes\ I think he should know if he*s going to 
be called. If I think the parent is not going to call him the answer is 'no' 
because it depends on the degree of unhappiness. (Principal, Site LUl) 

Clearly this principal's motives for approaching the superintendent were based on 
protection (for himself and for his superintendent). His insistence on maintaining an 
autonomous exist^nc? in his role effectively precluded his willingness to share 
information with <f a^. ais r over virtually all issues with the exception of those that 
became visible to the af.j/.ajser for various reasons. 

Principals ;u>civation to improve their performance was found to combine with 
experience to influence processing of appraisal data. Both factors acted indirectly on 
such mental processing through their influence on principals' attitudes toward 
appraisal. However, there was a negative relationship (correlation) between exper ience 
and motivation. Motivated principals who were relatively inexperienced in the roie were 
found to have more favorable attitudes toward the appraisal. They were inclined to 
participate in the appraisal process by ensuring that the appraiser had all available 
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information relevant to their performance. The more common scenario was one where 
experienced principals had developed poor attitudes toward the process and were not 
typically motivated to seek information relevant to improving their performance. A 
principal's (Site LU2) comments regarding the purposes of the appraisal system show 
that he did not include personal professional development among the expected benefits of 
the process. 

One purpose would be to let him [superintendent] get some insight and 
understanding of What's happening at the school ... it gives him an 
opportunity to give some praise for work that's well done ... to make sure we 
are communicating. 

The superintendent' understood the principals narrow view and expectations of the 
appraisal process. In the following excerpt he compared two of his principals, the other 
previously designated as a high user of appraisal data. 

He'd really go through it [appraisal process] as an expectation because he has 
to go through it. I would think that that may be a greater tendency with [him] 
than say with [the principal at Site HU2]. I would see [her] going through it 
because she really feels that it would be worthwhile for her own growth. 

Mentioned above were positive influences on processing of appraisal data 
attributable to opportunities for ongoing communication throughout the year and 
meaningful, quality, end-of-year communication sessions. Both of these factors were 
attributable to the relative sophistication of the appraisal process and they both had 
direct and indirect effects on the seriousness with which appraisal data were considered. 
As shown in Figure 2, another aspect of communication that was influential was its 
positiveness. Appraisal feedback presented in a positive, constructive and reinforcing 
manner, rather than being highly critical and negative was more likely to be taken 
seriously. 

Learning from Appraisal Data 

Factors associated with the decision setting and in particular, with characteristics 
of principals, were found to have quite powerful effects on the extent to which appraisal- 
based learning took place (see Figure 3). Principals' motivation for professional growth 
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had a strong effect on this type of use (albeit indirect) through influencing attitudes 
toward the appraisal process. Principals who were genuinely motivated to improve their 
performance welcomed all opportunities for evaluative feedback and. in fact, were 
sometimes observed to stimulate such opportunities (see comments by the principal of 
Site HU2 on page I). The appraisal process was viewed as one of many ways in which 
such information could be obtained. They also tended to reflect about their performance 
and to engage in self-evaluation exercises. 

Principals' attitudes were also influenced by other variables that had an indirect 
impact on use. Again, experience in the role was found to have a negative or inverse 
influence through its effects on principals' needs for information. Where principals were 
relatively experienced, this need was found not to be as great and, as a consequence, (1) 
they tended to be less receptive to the appraisal process, (2) the process was usually less 
rigorous, and (3) there was comparatively little ongoing communication between 
appraisers and principals throughout the school year. An experienced principal's (Site 
HUD description of why he did not learn much from the appraisal process illustrates 
this trend. 

I would say the purpose was to identify for me areas for growth; areas where I 
could concentrate on improving my performance, and also very definitely to 
provide encouragement, and a few fuzzies and strokes I guess for the job I do ... 
there weren't too many areas identified for growth ... I had no input into it ... I 
would hope that this year there will be areas that would help me formulate a 
manager's letter (action plan]. 

It is interesting that even though this principal perceived the focus of the appraisal to be 
on his professional development, its lack of thoroughness and ongoing interchange 
appeared to preclude effective conceptual development. 

Use of ApDraisal Data for Decisionmaking 

As seen in Figure 4, there were comparatively few decision setting variables that 
influenced the extent to which principals based decisions on appraisal data. Only one of 
these - experience and working knowledge - was associated with the personal 
characteristics of principals. Information from .sources beyond the appraisal process were 
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found to derive from such variables as political concerns in the school system and 
principals* acquired knowledge from years of practical experience in the role. This 
information influenced appraisal-based decisionmaking favorably provided that it was 
congruent with appraisal data, or that it raised the significance of a particular decision. 

An example comes from one of the low use sites (LUl). Given feedback from the 
appraisal in conjunction with other information, the principal decided to "step up" 
efforts in improving relations with his local community. He encouraged teachers to 
report to parents positive as well as negative information about students, he "beefed up" 
the schoofs newsletter and held more information sessions for parents in the school. The 
decision to improve community relations was significant to him because (1) from his own 
knowledge of the community he sensed that a problem existed and (2) the director of his 
school district had identified relations with the community to be a system-wide priority. 
This information was found by the principal to be entirely consistent with the evaluative 
feedback given by the appraiser and the result was the decision to move in this direction. 

Other factors associated with the implementation of the appraisal process also had 
substantial impact on appraisal-based decisionmaking (see Figure 4). Persistent 
communication of information about a particular issue from an appraiser viewed to be 
credible will have a positive influence on use of this type if (1) it is communicated in a 
constructive and reinforcing fashion and (2) it is congruent with information that has as 
its source something external to the appraisal. Mentioned previously was another 
principalis (Site HU2) decision to pursue the certification required of supervisory officers 
based largely on continued encouragement from her superintendent. She described her 
perceptions of him and how his persistence influenced her decision. 

I trust [his] judgment. I think he's a pretty astute judge of people .... He's a 
clear thinker and says what he thinks, and I like that kind of relationship.... 
Yes he has continued to say *go ahead with the exams' .... He has certainly 
never let me drop it. 

The strength of influence of these variables became apparent when the principal was 
asked about the effect on her decision if her superintendent had not been viewed by her 
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as being credible. Her response was "Fd probably disregard [what he was saying] and go 
to what other people were saying ... that would likely influence me more". 

The influence of characteristics of the appraisal process and of characteristics of 
the decision setting not associated with the principal appeared to play an important role 
in influencing the use of appraisal data for decisionmaking As described above, this was 
not the case concerning the extent to which principals learned from the appraisal 
process. Personal characteristics were found to be much more predictive of such 
conceptual outcomes. 

Discussion and Conclusions 

Prior to an interpretation of the present results it is important to recognize the 
limitations of the data. The design of the study relied on triangulation to foster the 
validity of the data. Data were collected in multiple sites from multiple sources (i.e., 
interviews, archival data) within each site. Data collection was guided by a conceptual 
framework grounded in empirical evidence but allowances were made for the emergence 
of unanticipated variables or relationships among variables. Further, interview data 
were collected based on knowledge from previous interviews. These design features 
support arguments for acceptable levels of internal validity but do not speak to issues of 
external validity (generalizability) or issues of reliability. Indeed, reliability is brought 
into serious question by the fact that only a single researcher (the present author) 
analysed the data. Replication of the findings through further research is one way of 
estimating the reliability of the data. Replication using different methods of data 
collection and analysis would help to establish the external validity or generalizability of 
the results (Brinberg & McGrath, 1982). In the absence of such evidence the reader 
must recognize the tentativeness of interpretations made here. With these considerations 
in mind, the following interpretations of the data are made. 

Results of the present study largely confirm in the context of personnel evaluation 
previous findings about the use of evaluation defined more broadly (Cousins & 
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Leithwood, 1986). In each of the two studies, factors associated with both the evaluation 
implementation and the decision or policy context were found to predict instrumental 
(decision oriented) uses of data. In the Cousins-Leithwood study (1) methodological 
attributes of the evaluation, (2) characteristics of the evaluation results, and (3) 
characteristics of the decision or problem focus for the evaluation were found to be the 
most prevalent factors influencing utilization as decision. Data from the present study 
suggested that the positive nature of communication of appraisal results and the 
congruence of those results with principals' expectations had moderate, positive 
influences on principals' use of data for decision. Also important were decision context 
variables corresponding to principals' experience and working knowledge and the 
significance of the decision for the principal. 

Evaluation-based conceptual development (learning) outcomes appear to be more 
dependent on attributes of the users or decisionmakers in both studies. According to 
Cousins and Leithwood, 

Evaluation quality was still found to be the most prevalent factor affecting 
this type of use. However, user commitment and/or receptiveness to the 
evaluation was the second most influential factor in determining the extent of 
staff or user conceptual development. Three factors, political climate, 
competing information, and personal characterisitics ... represented the next 
most influential factors at this level ... (1985, p. 358) 

In the present study, (1) principals' motivation for professional growth and (2) attitude 
twoard the appraisal were the most influential factors on learning about performance 
improvement. This dependent variable was also influenced by principals' experience and 
working knowledge and needs for information. These findings suggest that a suitable 
interpretation of the data is one that relies on conceptions of information processing 
characteristics of the users. Interpretation of this type is further justified by its 
consistency with current conceptions of the utilization process described earlier (e.g., 
Kennedy, 1984; Huberman, 1987). Such an approach to interpretation is described 
below. 
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Performance Improvement as Problem Solving 

Considerable research on human information processing has inquired about how 
individuals solve problems. Cognitive scientists have examined human problem solving 
capabilities in a wide range of domains including managerial information processing. 
Ungson, Braunstein and Hall (1981) provided a comprehensive review of research on 
managerial information processing and concluded that research in the area was moving 
in favorable directions, particularly regarding the selection of problems to be solved. 

Information processing research has undergone an evolution in the past two 
decades from a research area focusing on well-structured problems in highly 
controlled settings to one examining ill-structured problems that more 
accurately reflect organizational decision making, (p. 130) 

Leithwood anH Stager (1986, 1987) made similar observations about information 
processing literature. By applying an information processing framework to the study of 
principals' problem solving, they examined differences between high and moderately 
effective principals in solving "most-structured" and "least-structured" problems. They 
acknowledged that principals do face many well-structured problems solved by choosing 
from a set of known alternatives and basing choice on readily available decision rules. 
They also argued for the prevalence of less-structured problems facing principals. 

... school administrators also face "messy" situations in which they must first 
spend considerable effort in clarifying just what is the problem and what 
values are at stake in the solution. When such clarification has been achieved, 
there remains uncertainty about the goals to be accomplished, what 
constitutes an adequate solution process, and the nature of the constraints or 
obstac!?^ to solution that are likely to be encountered. (1987, p. 3) 

Due to a lack of readily available solutions and defensible criteria for choosing 
among them, personal professional development may be viewed by principals as an ill- 
structured problem to be solved. In this context, the use of appraisal data concerning 
their own performance is a solution process or strategy taken with the goal of solving the 
problem of performance improvement. Accordingly, the findings of Leithwood and 
Stager (1986, 1987) provide a interesting base from which to make comparisons. This is 
particularly the case since their research culminated in the development of a grounded 
framework for describing principals' problem solving processes. 
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Several findings from Leithwood and Stager's (1986) initial study are relevant to 
the present discussion. They concluded that the problem solving strategies of effective 
principals are characterized by an overall style that provides a central role in problem 
solving for others (e.g., peers, staff) and greater effort to systematically collect data. 
These principals were described as being willing to take risks in defining problems but to 
solve them in a comparatively risk-free style. In the present study relatively high users 
of appraisal data were shown to take an .cive role in their own appraisals by fostering 
the collection of information from a wide variety of sources and to ensure that such data 
were taken into consideration by appraisers. They also risked sharing information about 
their performance with their appraiser with the goal of coming to a more complete 
understanding of their own professional development needs. Low users of appraisal data, 
on the other hand, tended to take such action for reasons unrelated to their own 
development needs. G)mpared with their high use counterparts, they were **risk-averse" 
about the disclosure of personal performance information with their supervisors and 
were not inclined to systematically collect relevant information. 

In their 1987 paper, Leithwood and Stager developed a grounded framework 
describing the components of principals' problem solving. Five main components were 
identified: (1) interpretation , principals' understanding of what is the nature of the 
problem; (2) goals , the immediate purposes principals try to achieve in response to their 
interpretation of a problem; (3) principles , the long term purposes, values, fundamental 
laws, doctrines and assumptions guiding principals' thinking; (4) constraints , the 
immovable barriers, obstacles and factors severely narrowing the range of possible 
solutions principals believe to be available; and (5) solution processes , what principals 
do to solve problems in light of their interpretation, principles, goals and perceived 
constraints. Variations within each component were attributable to effectiveness in the 
role for both most* and least-structured problems. 

Effective principals were found to view least structured problems as solvable if 
careful thinking was used. They were very explicit about assumptions and used them for 
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interpretation in addition to past experience and information collection. Effective 
principals were also better able to arrive at a clear comprehensive interpretation of the 
problem allowing them to get on with the solution. Variations in interpretation of 
performance improvement problems by relatively high and low users of appraisal 
information, were apparent in data from the present study. Higher needs for information 
about personal performance were noted for a principal in the high use category who had 
comparatively less experience and working knowledge about the role of principal. High 
users of appraisal data tended to operate under the assumption that the appraisal 
process held potential value for them in fostering their professional growth and they 
assumed that there was room for improvement. Principals in the low use category, on 
the other hand, did not make these assumptions. They tended to equate greater 
experience in the role with effectiveness in school administration. Recognition that 
there was room for growth was downplayed and their appraisals were assumed to benefit 
others (e.g., enabling supervisor to keep tabs on a school). 

Effective principals were found to pursue a broader range of goals and to be more 
concerned with knowledge than with the feelings of others according to Lioithwood and 
Stager. The present study did not address, in an explicit way, the principals' goals nor 
their effect on use. There was modest evidence to show that educational goals had some 
influence on the significance of specific decisions for which appraisal information was 
available. The significance of decisions was also raised by external factors such as 
community concerns or appraiser persistence. The claim that this was more the case for 
low appraisal data users received mild support. 

Data from the present study about longer term goals or principles were equally 
indirect here. However, principals' drive and motivation for professional growth was 
found to be highly predictive of appraisal-based learning. One interpretation of this 
finding is that principals in the high use category had principles that were (1) more 
defensible and (2) they helped develop a better understanding of development needs. This 
assertion is consistent with Leithwood and Stager's findings concerning problem solving 
processes of effective principals. 
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The tendency to regard appraisals as being insufficient in rigor, credibility and 
ongoing communication was more typical of principals in the low use group. In 
situations where principals stimulate much of the data collection and interaction with 
supervisors, these appraisal process deficiencies might be interpreted as sub-problems to 
be solved rather than constraints. Data from Leithwood and Stager concerning the 
solving of least-structured problems suggested that only moderately effective principals 
view such problems to be constraints that either impair or seriously restrict successful 
problem solving. 

Previously described patterns of information collection, initiation of interaction 
with supervisors, self-evaiuation and reflection typical of principals that were relatively 
high users of appraisal data are consistent with Leithwood and Stager's observations 
about solution processes adopted by effective principals for least-structured problems. 
They described such processes as being well thought-out, planned, involving higher 
levels of consultation with others, and having greater emphasis on information collection 
for least-structured problems. 

ImDlications 

The interpretation of performance improvement as an ill-structured problem to be 
solved and the use of appraisal data as a solution process reveals a significant overlap 
between knowledge about principals that are relatively high users of appraisal data and 
knowledge about expert principal problem solving. This suggests that continued research 
about utilization would do well to be grounded in a human information processing 
framework. The work of Kennedy (1983, 1984) and of Hubeman (1987), for example, 
seems promising because of attention paid not only to how data-based knowledge 
structures are influenced but to the influence such structures have in their own right. 
Kennedy showed how such knowledge structures can compete with and, consequently, 
minimize the use of external evidence. But as external information is processed, 
knowledge structures can, in turn, be altered and a research report, for example, can be 
seen to have impact defined in these terms. 
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The conceptual model assumes these participants approach new evidence with 
an already-existing body of working knowledge, and that they use the evidence 
primarily to modify that knowledge. (Kennedy, 1984, p. 210) 

If I am an ^^instrumentalist'*, I am assuming a more passive, "reproductive*' 
user of research information**.* From a transactional or conflict-theoretic 
perspective, users are active "strategists who transform information to preserve 
cognitive consonance *** (Huberman, 1987, p* 590) 

The conceptual link to principal effectiveness made here carries with it 
implications for appraisal practice. An argument for diversification in appraisal system 
design seems appropriate. Appraisal systems designed for principals who are expert 
problem solvers might do well to concentrate on improving aspects of the appraisal itself 
such as rigor and communication quality in order to foster appraisal data use for 
performance improvement. These appear to be sub-problems whose solution might 
enhance use. Systems designed for moderately effective principalsy on the other hand, 
might be more concerned with decision setting variables that operate to negatively 
influence use, particularly ones associated with the characteristics of principals 
(motivation, attitiude, experience). It seems apparent that these are variables that will 
preclude effective use of appraisal data regardless of the characteristics of the appraisal 
or evaluation system. Improving the condition or state of some of these factors may well 
foster the use of performance appraisal for systematic and effective professional 
development. 
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